Generating intelligible audio speech from visual speech

This work is concerned with generating intelligible audio speech from a video of a person talking. Regression and classification methods are proposed first to estimate static spectral envelope features from active appearance model (AAM) visual features. Two further methods are then developed to incorporate temporal information into the prediction - a feature-level method using multiple frames and a model-level method based on recurrent neural networks. Speech excitation information is not available from the visual signal, so methods to artificially generate aperiodicity and fundamental frequency are developed. These are combined within the STRAIGHT vocoder to produce a speech signal. The various systems are optimised through objective tests before applying subjective intelligibility tests that determine a word accuracy of 85% from a set of human listeners on the GRID audio-visual speech database. This compares favourably with a previous regression-based system that serves as a baseline which achieved a word accuracy of 33%

Using Visual Speech Information in Masking Methods for Audio Speaker Separation

This work examines whether visual speech infor- mation can be effective within audio masking-based speaker separation to improve the quality and intelligibility of the target speech. Two visual-only methods of generating an audio mask for speaker separation are first developed. These use a deep neural network to map visual speech features to an audio feature space from which both visually-derived binary masks and visually- derived ratio masks are estimated, before application to the speech mixture. Secondly, an audio ratio masking method forms a baseline approach for speaker separation which is extended to exploit visual speech information to form audio-visual ratio masks. Speech quality and intelligibility tests are carried out on the visual-only, audio-only and audio-visual masking methods of speaker separation at mixing levels from -10dB to +10dB. These reveal substantial improvements in the target speech when applying the visual-only and audio-only masks, but with highest performance occurring when combining audio and visual information to create the audio-visual masks

Experimental Study of Parametric Autoresonance in Faraday Waves

The excitation of large amplitude nonlinear waves is achieved via parametric autoresonance of Faraday waves. We experimentally demonstrate that phase locking to low amplitude driving can generate persistent high-amplitude growth of nonlinear waves in a dissipative system. The experiments presented are in excellent agreement with theory.Comment: 4 pages, 4 eps figures, to appear in Phys. Rev. Let

Globalization, the ambivalence of European integration and the possibilities for a post-disciplinary EU studies

Using the work of Manuel Castells as a starting point, this article explores the ambivalent relationship between globalization and European integration and the variety of ways in which the mainstream political science of the EU has attempted to deal with this issue. The analysis here suggests that various 'mainstreaming' disciplinary norms induce types of work that fail to address fully the somewhat paradoxical and counter-intuitive range of possible relationships between globalization and European integration. The article explores critically four possible analytical ways out of this paradox—abandonment of the concept of globalization, the development of definition precision in globalization studies, the reorientation of work to focus on globalization as discourse, and inter- and post-disciplinarity. The argument suggests that orthodox discussions of the relationship require a notion of social geography that sits at odds with much of the literature on globalization and while greater dialogue between disciplines is to be welcomed, a series of profound epistemological questions need to be confronted if studies of the interplay between global and social process are to be liberated from their disciplinary chains

Platformed antagonism: Racist discourses on fake Muslim Facebook pages

This research examines how fake identities on social media create and sustain antagonistic and racist discourses. It does so by analysing 11 Danish Facebook pages, disguised as Muslim extremists living in Denmark, conspiring to kill and rape Danish citizens. It explores how anonymous content producers utilise Facebook’s socio-technical characteristics to construct, what we propose to term as, platformed antagonism. This term refers to socio-technical and discursive practices that produce new modes of antagonistic relations on social media platforms. Through a discourse-theoretical analysis of posts, images, ‘about’ sections and user comments on the studied Facebook pages, the article highlights how antagonism between ethno-cultural identities is produced on social media through fictitious social media accounts, prompting thousands of user reactions. These findings enhance our current understanding of how antagonism and racism are constructed and amplified within social media environments

Migraine aura: retracting particle-like waves in weakly susceptible cortex

Cortical spreading depression (SD) has been suggested to underlie migraine aura. Despite a precise match in speed, the spatio-temporal patterns of SD and aura symptoms on the cortical surface ordinarily differ in aspects of size and shape. We show that this mismatch is reconciled by utilizing that both pattern types bifurcate from an instability point of generic reaction-diffusion models. To classify these spatio-temporal pattern we suggest a susceptibility scale having the value [sigma]=1 at the instability point. We predict that human cortex is only weakly susceptible to SD ([sigma]<1), and support this prediction by directly matching visual aura symptoms with anatomical landmarks using fMRI retinotopic mapping. We discuss the increased dynamical repertoire of cortical tissue close to [sigma]=1, in particular, the resulting implications on migraine pharmacology that is hitherto tested in the regime ([sigma]>>1), and potentially silent aura occurring below a second bifurcation point at [sigma]=0 on the susceptible scale

Two new rapid SNP-typing methods for classifying Mycobacterium tuberculosis complex into the main phylogenetic lineages

There is increasing evidence that strain variation in Mycobacterium tuberculosis complex (MTBC) might influence the outcome of tuberculosis infection and disease. To assess genotype-phenotype associations, phylogenetically robust molecular markers and appropriate genotyping tools are required. Most current genotyping methods for MTBC are based on mobile or repetitive DNA elements. Because these elements are prone to convergent evolution, the corresponding genotyping techniques are suboptimal for phylogenetic studies and strain classification. By contrast, single nucleotide polymorphisms (SNP) are ideal markers for classifying MTBC into phylogenetic lineages, as they exhibit very low degrees of homoplasy. In this study, we developed two complementary SNP-based genotyping methods to classify strains into the six main human-associated lineages of MTBC, the 'Beijing' sublineage, and the clade comprising Mycobacterium bovis and Mycobacterium caprae. Phylogenetically informative SNPs were obtained from 22 MTBC whole-genome sequences. The first assay, referred to as MOL-PCR, is a ligation-dependent PCR with signal detection by fluorescent microspheres and a Luminex flow cytometer, which simultaneously interrogates eight SNPs. The second assay is based on six individual TaqMan real-time PCR assays for singleplex SNP-typing. We compared MOL-PCR and TaqMan results in two panels of clinical MTBC isolates. Both methods agreed fully when assigning 36 well-characterized strains into the main phylogenetic lineages. The sensitivity in allele-calling was 98.6% and 98.8% for MOL-PCR and TaqMan, respectively. Typing of an additional panel of 78 unknown clinical isolates revealed 99.2% and 100% sensitivity in allele-calling, respectively, and 100% agreement in lineage assignment between both methods. While MOL-PCR and TaqMan are both highly sensitive and specific, MOL-PCR is ideal for classification of isolates with no previous information, whereas TaqMan is faster for confirmation. Furthermore, both methods are rapid, flexible and comparably inexpensive

Proteome Sampling by the HLA Class I Antigen Processing Pathway

The peptide repertoire that is presented by the set of HLA class I molecules of an individual is formed by the different players of the antigen processing pathway and the stringent binding environment of the HLA class I molecules. Peptide elution studies have shown that only a subset of the human proteome is sampled by the antigen processing machinery and represented on the cell surface. In our study, we quantified the role of each factor relevant in shaping the HLA class I peptide repertoire by combining peptide elution data, in silico predictions of antigen processing and presentation, and data on gene expression and protein abundance. Our results indicate that gene expression level, protein abundance, and rate of potential binding peptides per protein have a clear impact on sampling probability. Furthermore, once a protein is available for the antigen processing machinery in sufficient amounts, C-terminal processing efficiency and binding affinity to the HLA class I molecule determine the identity of the presented peptides. Having studied the impact of each of these factors separately, we subsequently combined all factors in a logistic regression model in order to quantify their relative impact. This model demonstrated the superiority of protein abundance over gene expression level in predicting sampling probability. Being able to discriminate between sampled and non-sampled proteins to a significant degree, our approach can potentially be used to predict the sampling probability of self proteins and of pathogen-derived proteins, which is of importance for the identification of autoimmune antigens and vaccination targets

Affine differential geometry analysis of human arm movements

Humans interact with their environment through sensory information and motor actions. These interactions may be understood via the underlying geometry of both perception and action. While the motor space is typically considered by default to be Euclidean, persistent behavioral observations point to a different underlying geometric structure. These observed regularities include the “two-thirds power law” which connects path curvature with velocity, and “local isochrony” which prescribes the relation between movement time and its extent. Starting with these empirical observations, we have developed a mathematical framework based on differential geometry, Lie group theory and Cartan’s moving frame method for the analysis of human hand trajectories. We also use this method to identify possible motion primitives, i.e., elementary building blocks from which more complicated movements are constructed. We show that a natural geometric description of continuous repetitive hand trajectories is not Euclidean but equi-affine. Specifically, equi-affine velocity is piecewise constant along movement segments, and movement execution time for a given segment is proportional to its equi-affine arc-length. Using this mathematical framework, we then analyze experimentally recorded drawing movements. To examine movement segmentation and classification, the two fundamental equi-affine differential invariants—equi-affine arc-length and curvature are calculated for the recorded movements. We also discuss the possible role of conic sections, i.e., curves with constant equi-affine curvature, as motor primitives and focus in more detail on parabolas, the equi-affine geodesics. Finally, we explore possible schemes for the internal neural coding of motor commands by showing that the equi-affine framework is compatible with the common model of population coding of the hand velocity vector when combined with a simple assumption on its dynamics. We then discuss several alternative explanations for the role that the equi-affine metric may play in internal representations of motion perception and production